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EXECUTIVE SUMMARY 



Due to the isolation and limited resourees that rural sehools often faee, retaining teaehers in rural 
settings is partieularly ehallenging. One way to reduee the sense of isolation within the teaching 
profession is to develop a professional community. Research suggests that collective learning 
and peer support are critical if teacher professional development is to result in changes in teacher 
practice, but meeting this need can be difficult in rural school districts that are small and isolated. 
Technology provides one way to address the professional development needs of rural teachers. 

This study investigated McREL’s Rural Technology Initiative (RTI), an online professional 
development intervention that provides opportunities for teacher collaboration while delivering 
training to teachers and administrators on instructional improvement, including the use of 
technology for teaching and learning. The purposes of the study were: (1) to examine the effects 
of online professional development on teacher professional community and teacher retention, (2) 
to evaluate the influences of the RTI on teacher instruction, teacher and administrator use of 
technology, and administrator practice, and (3) to identify those aspects of the RTI intervention 
that worked best and those that should be changed. 

From a pool of 22 schools, McREL researchers chose seven pairs to participate in the study, 
based on the extent to which their rural status, demographic characteristics, and technology 
environments matched. Within each pair, schools were randomly assigned to the RTI treatment 
group or to the no-treatment control group. The study sample included 126 teachers and 14 
administrators from six treatment schools (one school withdrew from the study) in four states, 
and 136 teachers and 17 administrators from seven control schools in three states. 

The RTI was implemented between April 2004 and May 2005. The main components of the 
intervention were two monthly, one-hour online classes for teachers on instructional strategies, 
monthly threaded online teacher discussions, and one monthly online meeting for administrators 
on topics related to leadership. 

Data were collected through teacher and administrator surveys administered before and after the 
RTI, an administrator interview and survey, and two focus groups. In addition, two of the online 
threaded discussions were analyzed for content. The study addressed six research questions: 

1 . What is the effect of the RTI on teacher professional community? 

2. What is the effect of the RTI on variables related to teacher retention, 
including teaching efficacy, job satisfaction, and intention to stay in teaching? 

3. What is the influence of the RTI on teachers’ use of effective instructional 
practices? 

4. How effective is the RTI at increasing teacher and administrator comfort with 
and use of technology? 

5. What is the influence of the RTI on administrators’ leadership practice? 

6. What aspects of the RTI intervention did participants find effective and what 
improvements should be made? 
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The data indicated that the RTI was implemented as planned with relatively few technological 
difficulties. However, teacher participation in RTI activities varied considerably, with those who 
participated more perceiving more impact on their instruction and their students’ learning. 



Together the various sources of data and the analyses of these data led to the following 
conclusions: 

• The RTI had a positive influence on teacher collaboration and the 
development of professional community. 

• The RTI did not influence teachers’ perceptions of their teaching efficacy, job 
satisfaction, or school environment. 

• Teachers who participated more in the RTI benefited more in instructional 
improvements. 

• There was limited evidence that the RTI increased technology use though this 
was mainly because the schools that participated were already technologically 
proficient. 

• Teachers did not perceive changes in their principals’ leadership, but 
administrators perceived positive influences of the RTI on their leadership 
practices. 

• Teachers and administrators were satisfied with the content and delivery of 
the RTI professional development. 

The evaluation data from the study suggested several recommendations for improvement of the 
RTI in the areas of scheduling, participant information, discussion boards, and logistics. 
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OVERVIEW 



In 1996, the National Commission on Teaching and America’s Future (NCTAF) documented the 
influence of teachers on improvements in student learning and on the success of standards-based 
reforms. In 2003, the commission again stressed the need to provide every child in America with 
a qualified teacher, but lamented that efforts to achieve this goal have been undermined by high 
rates of teacher turnover and attrition, particularly in low-income communities and rural areas. 
According to NCTAF, one way to address this problem is to build a high-quality teaching 
profession that increases teachers’ job satisfaction. Among the strategies that NCTAF 
recommends for accomplishing this goal is to support both new and experienced teachers 
through professional development. 

This document reports research and evaluation findings from a study of an online professional 
development intervention for rural teachers and administrators. The sections that follow describe 
the background, rationale and study questions, method, results, and conclusions. The audience 
for this report includes education researchers, K-12 administrators, staff developers, and 
technology directors. Policymakers in rural states also may find the report informative. 

BACKGROUND 



Teacher Retention 

The U.S. Department of Education projects that by the year 2010 there will be a demand for two 
million teachers (Moir & Gless, 2001). To an extent, this demand depends on increasing student 
enrollment, but it is primarily a result of high rates of teacher turnover (Ingersoll, 2001). 
According to the 1999-2000 Schools and Staffing Survey, 14 percent of new teachers leave after 
their first year of teaching, and another 15 percent transfer to a different school (McClure, 
Redfield, & Hammer, 2003). Teachers leave for a variety of reasons, including inadequate 
preparation; conditions in the school and classroom such as a lack of support, discipline 
problems, and lack of influence over decisions; family or personal reasons; retirement; other job 
opportunities; and dissatisfaction with salaries and benefits (Southern Regional Education Board, 
2004; Ingersoll, 2001). NCTAE (2003) concluded that teacher turnover affects a school’s 
financial position, improvement efforts, teacher quality, and student achievement. In schools 
with high teacher turnover, there is less opportunity to develop a cohesive, collaborative staff 
than in schools with low teacher attrition. Staff development becomes more challenging because 
different teachers are continually passing through the system, making sustained professional 
development difficult. Students also suffer because they are denied experienced teachers and are 
offered fewer course options (Emerson, 2003; Chambers, 2000). 

Due to the isolation and limited education resources and funding that rural schools often face, 
retaining teachers in rural settings is particularly challenging. In a survey of superintendents of 
rural school districts, low salaries, social isolation, and geographic isolation were the most 
frequently cited obstacles to attracting and retaining teachers. Additional barriers to hiring and 
retaining teachers were the need to teach multiple subjects, limited opportunities for training, and 
the poor economic health of the surrounding community (Schwartzbeck, Redfield, Morris, & 
Hammer, 2003). Teachers in rural areas generally are paid less than teachers in non-rural areas, 
and the disparities in salaries between rural and non-rural districts within the same state make it 
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difficult for rural communities to retain teachers (Jimerson, 2003; McCreight, 2000). Strategies 
suggested for retaining rural teachers include increased salary and benefits, collegial mentoring, 
support for beginning teachers, opportunities for professional development, and integrating 
technology into classrooms and professional development (Collins, 1999; McCreight, 2000; 
Harmon, 2001; Schwartzbeck et ah, 2003). 

Collaboration through Professional Community 

If one of the main reasons teachers leave rural areas is professional isolation (Collins, 1999; 
Chambers, 2000; Hillkirk, Chang, Oettinger, Saban, & Villet, 1997), then decreasing teachers’ 
sense of isolation might increase teachers’ job satisfaction and thereby influence the decision to 
stay in rural areas. One way to reduce the sense of isolation within the teaching profession is to 
develop a professional community. 

Newmann and his colleagues described five aspects of professional community in their research 
on restructuring schools (Newmann & Wehlage, 1995; Louis, Marks, & Kruse, 1996; Louis, 
Kruse, & Marks, 1996). According to these researchers, a professional community is defined by 
shared norms and values, collective focus on student learning, collaboration, deprivatization of 
practice, and reflective dialogue. Professional community among teachers is associated with 
variables related to student achievement, such as authentic pedagogy and instruction and 
teachers’ shared responsibility for student learning (Louis & Marks, 1998) and with variables 
related to teachers’ perceptions of their work, such as a sense of affiliation and support (Louis, 
Marks, & Kruse, 1996; Secada & Adajian, 1997; Newmann & Wehlage, 1995). Teachers in 
professional communities are encouraged to work together in one another’s classrooms and to 
interact by observing, mentoring, providing feedback, and sharing expertise (Louis, Marks, & 
Kruse, 1996; Bryk, Camburn, & Louis, 1999). This deprivatization of practice not only decreases 
teachers’ professional isolation but also encourages teachers to think about, analyze, and share 
knowledge related to instruction, curriculum, and student learning, which leads to a deeper 
understanding of teaching (Louis, Marks, & Kruse, 1996; Secada & Adajian, 1997). 

Research suggests that collective learning and peer support are important aspects of teacher 
professional development if it is to result in changes in teacher practice (Desimone, Porter, 

Caret, Yoon, & Birman, 2002; Schlager, 2003; Joyce & Showers, 1995). Meeting this need can 
be a challenge for rural school districts that are small and isolated, as indicated in a report on 
teacher quality from the National Center for Education Statistics (1999). Table 1 shows the 
percentage of teachers in areas and cities of differing sizes who reported that they never 
participated in different types of collaborative professional development. 
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Table 1. Percentage of Teachers Reporting that They Never Participated in Collaborative 
Professional Development of Varying Formats 



Topic 


Type of Town or City 


Rural/ 
Small Town 


Urban Fringe/ 
Large Town 


Central City 


Never participated in... 


Common planning time 


44% 


35% 


34% 


Regularly scheduled collaboration with other 
teachers, excluding meetings held for 
administrative purposes 


22% 


19% 


16% 


Being mentored in a formal relationship 


82% 


82% 


78% 


Mentoring another teacher in a formal 
relationship 


78% 


74% 


71% 


Networking with teachers outside the school 


46% 


36% 


34% 



Source; National Center for Education Statistics (1999) 



The same report indicates that rural teachers generally have fewer opportunities for professional 
development than teachers in other locations. Table 2 shows the percentage of teachers in 
locations of differing sizes who, for varying topics, reported that in the past 12 months they had 
not participated in professional development. 



Table 2. Percentage of Teachers Reporting No Participation in Professional Development 
on Varying Topics in Past 12 Months 



Topic 

Never participated in... 


Type of Town or City 


Rural/ 
Small Town 


Urban Fringe/ 
Large Town 


Central City 


In-depth study in the subject area of your 
main teaching assignment 


30% 


28% 


24% 


New methods of teaching 


25% 


23% 


22% 


Student performance assessments 


36% 


31% 


31% 


Classroom management 


53% 


53% 


47% 


Addressing the needs of English language 
learners or students from diverse cultural 
backgrounds 


82% 


67% 


58% 



Source: National Center for Education Statistics (1999) 



The Role of Technology in Professional Development 

Technology provides a way to address the professional development needs of rural teachers. 
Online professional development is a method for providing teachers with training and peer 
support, especially the teachers who are physically isolated (Mather, 2000). Although the 
research on outcomes of online professional development is limited, there are studies that 
suggest its potential for supporting teacher learning. For example, a study that examined 
telecollaborative technology projects using asynchronous online forums to connect students. 
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scientists, and teachers found that teachers experienced substantial benefits from their online 
communications (Hawkes & Good, 2000). There was a significant increase in participating 
teachers’ perceptions of the degree to which education technology increased their instructional 
practices, including their abilities to develop curricula, manage learning resources, and teach 
with an interdisciplinary focus. Another study examined the Rural Telecomputing Initiative, 
which created online professional development activities to connect rural math and science 
teachers. An online database of science and math resources was available to teachers, and 
teachers also were able to communicate with peers and experts through online networks. 
Research indicated that the program reduced teachers’ professional isolation and increased their 
collaboration with colleagues and experts, including those outside of their education 
communities (Yap, 1997). In a study of TAPPED IN, a virtual professional development 
community, teachers who logged in more, and therefore participated more, reported less 
professional isolation and more impact on their teaching practices than those teachers who 
logged in less (Fusco, Gehlbach, & Schlager, 2000). Heath and Yost (2001) described the 
Teacher Training Academy, a program that matched a skilled facilitator with small groups of 
teachers in a virtual community. Teachers’ positive perceptions of this program were related to 
the materials used for discussion and reflection, the variety of participants from around the 
country, the suggestions offered by colleagues, and the convenience of completing the activities 
at any time. 

A different kind of online staff development program offered through West Virginia University 
used video, asynchronous discussion groups, and virtual bulletin boards to offer staff 
development for in-service training. Participants found the delivery of web-based staff 
development to be an effective and efficient way to disseminate information, especially to 
practitioners in rural areas (Ludlow, 2002). Survey results showed that teachers increased their 
confidence in working with computers, and teachers in focus groups reported learning new 
content and technical skills. 

Finally, in a report for the Southern Regional Education Board, Thomas (2004) advocated for 
states’ use of online professional development, citing as a key benefit connections among 
teachers who would not be in touch with one another without the online environment. Thomas 
also described a study in Baltimore in which teachers in an online technology course reported 
having a more positive experience than those who took the same course in a face-to-face setting. 
In explaining their reactions, the online participants cited their comfort in giving information and 
receiving feedback from others in the course. 

In summary, there are a number of benefits to online professional development programs in 
general and especially for those living in rural areas. Teachers have access to resources that 
might not be available locally. Teachers gain experience with using technology for learning and 
presumably increase their understanding of how to integrate technology into their own teaching 
practices. Online professional development is ongoing and extended, so teachers are continually 
learning and building upon what they learn. Technology also provides a way for teachers to 
connect with colleagues, including those who are physically distant. Finally, participants can 
receive feedback and learn about different perspectives from a variety of online peers who have 
different backgrounds or experiences. 
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The Role of Administrators in Professional Development 



Whether it is delivered online or face-to-face, research suggests that support from school 
administrators is an important aspect of teacher professional development. Bredeson and 
Johansson (2000) conducted 48 structured interviews of principals, school administrators, and 
teachers to examine the range of school principal roles related to teacher professional 
development. The researchers describe principals as both leaders and learners. Principals are 
stewards who value learning for all those in the school, and they are models who participate with 
teachers in professional development efforts. Principals are experts with professional knowledge 
related to teaching and learning, and they are instructional leaders who guide the school in 
accomplishing its learning objectives and goals. Another obvious role of principals is to create a 
supportive learning environment for teacher learning, where teachers have the freedom to 
practice new ideas. 

Recent research in South Carolina demonstrates the relationship between teachers’ perceptions 
of working conditions and school leadership, including leaders’ support for teacher learning. 
Based on a survey of over 15,000 teachers, Hirsch (2004) reports, “Leadership is highly 
correlated with all working conditions, particularly professional development” (p. 25). The 
author explains that strong leaders involve teachers in decisions about professional growth and 
provide the resources and encouragement for teachers to work together. Importantly, Hirsch 
found correlations between teachers’ perceptions of leadership and both teacher retention and the 
accountability status of the teachers’ schools under NCLB. 

Others have stressed the importance of administrators in moving schools toward technology 
integration. In a report on technology in U.S. schools, Coley, Cradler, and Engel (1997) 
observed, “Research on the adoption of innovations in schools consistently points to the key role 
of administrative leaders in the successful implementation of innovations” (p. 46). Coley et al. 
described approaches for training teachers in technology implementation in which principals and 
other administrators also participate. Such participation helps principals to improve their own 
understanding of how to integrate technology and learning and also indicates to teachers the 
importance of staff development. Mize and Gibbons (2000) reported on three case studies of 
schools that were addressing technology integration. Based on teacher surveys and principal and 
teacher interviews, the researchers concluded that administrators’ involvement in and support of 
teacher training were important in moving the schools toward technology proficiency. 

To summarize, support from school leaders is needed for professional development to change 
teacher and school practices and to result in perceptions of positive working conditions. Leaders 
can demonstrate this support by participating in the training that teachers receive and by 
providing resources (e.g., time, encouragement) for teacher collaboration and reflection. 

Rationale 

The previous sections described the problem of teacher retention and why it poses challenges for 
rural schools. Research on why teachers leave rural schools suggests that a primary reason is 
isolation, both physical and professional. A way to reduce teachers’ sense of isolation in their 
profession is through participation in a professional community. Development of professional 
community requires reflective dialogue, deprivatization of practice, and collaboration with other 
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teachers teaching similar subject and grade levels, which can be difficult to implement in single 
rural schools due to their small size and physical isolation. Technology provides a possible 
solution. 

This study investigates McREL’s Rural Technology Initiative (RTI), an online professional 
development intervention. The RTI provides opportunities for participating in teacher 
professional communities within and across schools while delivering training to teachers and 
administrators on instructional improvement, including the use of technology for teaching and 
learning. The RTI also delivers training to administrators related to improved leadership 
practices. 

This study has three main purposes: (1) to examine the effects of online professional 
development on teacher professional community and teacher retention, (2) to evaluate the 
influences of the RTI on teacher instruction, teacher and administrator use of technology, and 
administrator practice, and (3) to identify those aspects of the RTI intervention that worked best 
and those that should be changed. These purposes are reflected in the research questions listed in 
the next section. 



METHOD 

This section describes the research questions, the intervention, the sample of schools and 
teachers, and the data collection instruments. 

Research Questions 

Table 3 indicates the research questions and the data sources for each question. 



Table 3. Research Questions and Data Sources 



Research 

Question 


Teacher 

Pre-post 

Intervention 

Survey 


Treatment Teacher 
Online Discussion 


Teacher & 
Administrator 
Pre-post 
Technology 
Surveys 


Treatment 
Administrator 
Satisfaction Survey 


Treatment 

Administrator 

Interview 


Focus Groups 


1 . What is the effect of the RTI 
on teacher professional 
community? 


X 


X 










2. What is the effect of the RTI 
on variables related to 
teacher retention including 
teacher efficacy, job 
satisfaction, and intention to 
stay in teaching? 


X 












3. What is the influence of the 
RTI on teachers’ use of 
effective instructional 
practices? 


X 






X 
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Research 

Question 


Teacher 

Pre-post 

Intervention 

Survey 


Treatment Teacher 
Online Discussion 


Teacher & 
Administrator 
Pre-post 
Technology 
Surveys 


Treatment 
Administrator 
Satisfaction Survey 


Treatment 

Administrator 

Interview 


Focus Groups 


4. How effective is the RTI at 
increasing teacher and 
administrator comfort with 
and use of technology? 


X 




X 


X 






5. What is the influence of the 
RTI on administrators’ 
leadership practice? 


X 




X 


X 






6. What aspects of the RTI 
intervention did participants 
find effective and what 
improvements should be 
made? 


X 






X 


X 


X 



Intervention 

The RTI is an online staff development intervention aimed at helping rural schools — especially 
those in more remote areas — provide staff development to teachers and administrators without 
the expense of travel and lodging. The goal of the RTI is to create a collaborative online network 
among rural and/or geographically isolated schools, providing online staff development 
regarding instruction and technology integration and helping administrators build an 
environment that fosters technology-rich instruction. 

A memorandum of understanding between McREL and each participating school specified that 
“80% or more of the teachers and administrators attend two online monthly meetings,” omitting 
the summer months. The RTI was implemented for this study between April 2004 and May 2005 
and included the following activities: 

• A face-to face orientation meeting of school representatives (an administrator 
and one teacher from each school) and McREL RTI implementers in April 
2004 

• Two monthly, one-hour online classes for teachers 

- The content of the general and second teacher meetings was drawn from 
Classroom Instruction That Works (Marzano, Pickering, & Pollock, 2001), 
which describes nine categories of research-based strategies for classroom 
instruction. The strategies are based on a meta-analysis of comparison- 
group research studies on the effects of instructional techniques on student 
achievement (Marzano, 1998). The average standardized effect sizes of 
the nine categories of strategies range from .59 to 1.61, indicating their 
potential for positive influences on student outcomes. A different strategy 
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was taught each month, except during December and January, which were 
combined due to the holiday breaks. 

- The first class concerned that month’s instructional strategy, and the 
second class focused on integrating technology with the instructional 
strategy presented in the first class. The technology strategies were based 
on an article by Brabec, Fisher, and Pitler (2004), which explains how 
teachers can use different technologies to support the instructional 
strategies described in Classroom Instruction That Works. Schools were 
given a choice of two times each month for teachers to attend the second 
follow-up class, which was designed to include brief times for schools to 
discuss the topics and report back to the large group. It was originally 
planned to separate teachers attending the second class into several live 
chat rooms based on teacher subject or grade; however, the chat rooms 
were discontinued in October based on administrator feedback concerning 
technical difficulties. 

- E-Campus software was used to distribute materials (i.e., PowerPoint and 
Word files) and support the threaded discussion boards. Materials posted 
on the E-campus site were available to participants during and after class 
meetings. These materials always included the meeting invitations and 
overview for each month and all PowerPoint presentations for each online 
class, as well as the discussion assignment and the threaded discussion 
board for each month. Additional materials were posted to E-Campus as 
needed and included samples from the instructional strategies and 
technology applications discussed in meetings; links to pertinent websites; 
and the articles, resources, and tools discussed in the classes. The video 
conferencing software was WebEx through mid- January and Marratech 
was used thereafter (starting in early February). 

- The instructors were trainers with expertise in technology, teaching, and 
the content of Classroom Instruction That Works. Three had published the 
article on technology strategies used in the online classes (Brabec et al, 
2004). The fourth instructor joined the team in 2005 and met with the RTI 
director to go over content and procedures and debriefed with the training 
team after each class. 

• Threaded online teacher discussions and debriefing with other teachers in the 
project. Online discussions using E-campus were designed to provide 
opportunities to develop professional community though engaging in 
reflective dialogue about instructional practice. The trainers provided initial 
prompts at the second class meeting, usually asking teachers to try out the 
strategies and report back. 

• One online meeting each month for administrators on topics related to 
leadership development and fostering innovation. Software was the same as 
that used for the teacher class meetings. Materials included Balanced 
Leadership: What 30 Years of Research Tells Us about the Effect of 
Leadership on Student Achievement (Waters, Marzano, & McNulty, 2003), 
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Good to Great (Collins, 2001), and First, Break all the Rules (Buckingham & 
Coffman, 1999). 

Design and Sample 

To identify the initial population of participating schools, McREL’s director of technology 
contacted the state director of technology in each of the seven Central Region states served by 
McREL. The state directors suggested names of rural schools that might benefit from the RTI. 
Schools also were identified from those that responded to a notice about the RTI posted on 
McREE’s website. The resulting initial population included 61 rural schools; 41 of these schools 
agreed to take the McREE Technology Surveys for teachers and administrators (Pitler, Enriquez- 
Olmos, & Manley, 2003), and 32 schools completed and returned the surveys. The surveys 
measure teachers’ and administrators’ perceptions of their technology skills and comfort level 
and the degree to which the school uses technology and integrates it with teaching and learning. 
The teacher response rate within schools ranged from 29 percent to 100 percent. Only the 22 
schools that had a teacher response rate of 70 percent or higher were considered for participation 
in the study. 

Erom the 22 schools, seven pairs of schools were chosen based on the degree to which they were 
matched on the following characteristics: rural status, state, school size, Title-I status, student 
ethnicity, student socioeconomic status, teachers’ perceived comfort with technology, and 
teachers’ perceptions of the school community’s acceptance of technology (Pitler et al, 2003). 
Schools in each pair were randomly assigned to the treatment/intervention group or to the 
control/comparison group. One treatment school chose not to participate and was switched with 
its matched control school, and a control school chose not to participate and was replaced with a 
school in the overall pool that had similar characteristics. An additional school was recruited to 
participate as a backup control site. In one district, the elementary school was a treatment site, 
and the middle school/high school was in the initial control group. Due to possible exposure to 
the RTI materials through the superintendent who participated in the RTI, this control middle 
school/high school was eliminated from the study. One treatment school dropped out of the RTI 
in August 2005 due to school management issues. Table 4 indicates the characteristics of the 
schools in the final study sample. 

Data Collection 

This section describes the instruments and procedures used to collect data from the sources 
shown in Table 3. Appendix A contains the survey items for the constructs addressed by the 
teacher and administrator surveys. The primary data collection instrument was a teacher pre-post 
intervention survey. Table 5 shows the constructs that the teacher survey addressed and gives 
examples of survey items. 
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Table 4. Characteristics of Treatment and Control Schools 



School Characteristic 


Treatment Schools 
(n = 6) 


Control Schools 
(n = 7) 


Rural status 


All Rural, outside of a 
Metropolitan Statistical Area 


All Rural, outside of a 
Metropolitan Statistical Area 


Geographic distribution 


KS, MO, NE, SD 


CO, NE, SD 


Grade distribution 


Pre-K-12 


Pre-K-12 


School Title I status 


5 schools 


7 schools 


Student ethnicity 


65% - 100% white 
Mean = 91% white 


49% - 100% white 
Mean = 92% white 


Student socioeconomic status 


45% Free /reduced lunch 


39% Free /reduced lunch 


Teacher perceived technology 
comfort* 


Mean rating = 2.47 


Mean rating = 2.65 


Teacher perceived acceptance 
of technology in the school ** 


Mean rating = 3.81 


Mean rating = 3.59 


Mean number of students per 
school 


205 


231 


Number of teachers 
participating in the study 


123 


136 


Number of administrators 
participating in the study 


14 


17 



*response set: 1 = very uncomfortable, 5 = very comfortable 
**response set: 1 = strongly disagree, 5 = strongly agree 



Data were collected to determine the extent to which the RTI influenced teacher professional 
community and teacher retention. The teacher pre-post intervention survey measured two aspects 
of teacher professional community — shared norms and values and teacher collaboration (Louis 
& Marks, 1998). Due to the short time span of the intervention (14 months), the primary data 
sources on retention were teachers’ self-reported intentions to stay in teaching and teachers’ 
perceptions of variables that were related to retention in prior research. The latter include job 
satisfaction, motivation for teaching, and teacher efficacy. Teachers’ dissatisfaction with their 
teaching jobs is associated both with teachers’ decisions to move to other schools and their 
decisions to quit the profession (Ingersoll, 2001). Sources of motivation for teaching, such as 
salary and interacting with children, are linked to job satisfaction and commitment to teaching 
(Dilworth, 1991). Finally, teachers with a higher sense of teaching efficacy are more committed 
to teaching and more likely to remain in teaching, as compared to teachers with a lower sense of 
teaching efficacy (Tschannen-Moran, Hoy, & Hoy, 1998). 

In an effort to more fully understand RTI influences on professional community and retention, 
data were collected on school environment and principal support based on evidence that both are 
needed for the development of professional community and effective teacher collaboration 
(Newmann & Wehlage, 1995; Hirsch, 2004; Van Buhler & Lauer, 2005). A final element 
measured by the teacher pre-post intervention survey was the amount of different types of 
professional development that teachers experienced the school year immediately prior to the RTI 
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and also other professional development they had during the RTI. These data were collected to 
examine possible influences from professional development other than the intervention. 



Table 5. Constructs on Teacher Pre-post Intervention Survey 



Construct 


Sample Survey Item 


Professional community- shared norms & 
values (Louis & Marks, 1998), n = 5 items 


Teachers share beliefs and values about what the 
central mission of this school should be. (agreement) 


Professional community- collaboration 
(Louis & Marks, 1998), n = 9 items 


Meeting with other teachers on lesson planning, 
curriculum development, or other collaborative work 
related to instruction (frequency) 


Intention to stay in teaching (Schools & 
Staffing Survey; NCES, 2001), n = 2 items 


I plan to remain in teaching or K-12 education as long 
as I’m working, (agreement) 


Job satisfaction (Schools & Staffing Survey; 
NCES, 2001), n = 4 items 


All things considered, I am satisfied with being a 
teacher, (agreement) 


Motivation for teaching (Davis, 2002), n = 17 
items 


Influence of salary and benefits, materials and 
resources, professional development opportunities, 
rural lifestyle/community, etc. (extent) 


Teacher efficacy (Tschannen-Moran et al., 
1998), n = 10 items 


When I really try, I can get through to most difficult 
students, (agreement) 


School environment- general support 
(Newmann & Wehlage, 1995; Charlotte 
Advocates for Education, 2004), n = 9 items 


Staff members are recognized for a job well done, 
(agreement) 


School environment- principal support (Eee & 
Smith, 1996; Charlotte Advocates for 
Education, 2004), n = 7 items 


Tbe principal is interested in innovation and new 
ideas, (agreement) 


Professional development (Schools & Staffing 
Survey; NCES, 2001), n = 5 items 


Attended professional development activities 
developed by your school (amount) 



In addition to the 74 survey items related to the constructs listed in Table 5, a total of 14 items 
addressed the teacher’s background, including education, teaching experience, and current 
teaching assignment — student grade levels, subject areas, classroom access to the Internet, and 
planning time. The post-intervention surveys for both treatment and control teachers included the 
items on the pre-survey and additional items related to the professional development that 
teachers received between the surveys. The teachers in the treatment schools were queried about 
their specific experiences with the RTI, including amount of participation, use of RTI 
instructional and technology strategies in their instruction, perceptions of change in their 
teaching and in their principal’s leadership, and satisfaction with different aspects of the RTI. 

Selected scales from McREL Technology Surveys for teachers and administrators (Pitler et al, 
2003) were used to measure treatment and control teachers’ and administrators’ perceptions and 
attitudes toward technology both before and after the implementation of the RTI. Additional 
items concerning the RTI were added to the post-technology surveys that were given to teachers 
and administrators participating in the treatment group. 
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Administrators in the treatment group completed a post-intervention survey about their 
satisfaction with the RTI. The survey measured administrators’ perceptions of the quality and 
usefulness of different RTI components, administrators’ and teachers’ attendance at the online 
classes, and perceptions of changes in their teachers’ instruction and their own leadership 
practices. 

All surveys were administered online using Perseus software. Each participant received an e- 
mail message that provided a URL link to the appropriate survey. In every case, the message 
explained that participation was voluntary, that answers were confidential, and that no individual 
teacher, administrator, school, or district would be identified in any reports about the data. As an 
incentive for completing the pre- and post- intervention surveys, teachers in both the treatment 
and control groups had a chance to win one of eight $50 online gift certificates (four per group) 
when they completed the surveys by the requested dates. 

In addition to the data collection from online surveys, treatment administrators participated in a 
telephone interview with a researcher midway through the RTI intervention. The interview 
protocol had formative evaluation questions about different aspects of the delivery and content of 
RTI classes and discussion groups, perceptions of teachers’ engagement, perceptions about 
effects from the RTI, and recommendations for improvement. 

Finally, researchers conducted face-to-face focus groups in June 2005 after the completion of the 
RTI to further explore participants’ experiences with the RTI and to identify problems and solicit 
recommendations. There was one focus group of teachers and one focus group of administrators. 

RESULTS 

This section begins with an examination of the degree to which the intervention occurred 
as planned. Next, there is a detailed description of the respondents for the pre-post 
intervention survey, followed by an analysis of treatment teachers’ participation in the 
RTI. The results related to each of the six research questions follow. The results section 
concludes with findings that are not directly connected with the research questions but 
which inform the interpretation of the results for these questions. 

Fidelity of Implementation 

All online meetings took place as planned. This set of meetings included 10 general teacher 
classes, eight follow-up teacher classes, and nine administrator meetings. Using an observation 
protocol, a researcher observed a sample of meetings to record how the technology was 
functioning, the degree to which discussions were on topic, and the degree of participant 
engagement. 

Six of the ten general teacher classes were observed. For the most part, these online classes were 
implemented as planned. However, in the February 2005 meeting, a change in video 
conferencing software resulted in one school being unable to connect and caused less severe 
interruptions and difficulties for several other participating schools. The second instance in 
which a school could not connect occurred in April 2005. Among the remaining four general 
classes observed, there were no technical difficulties or only glitches that were minor and did not 
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prevent school participation. Five of the six observed general classes included discussion that 
occurred mainly within each school. The wrap-up meeting was almost entirely whole group 
discussion. In all cases the discussions were considered to be “on-topic.” For the five meeting 
that included discussion, the observers rated participants as “very engaged” in three meetings, 
and “somewhat engaged” in two meetings. 

A researcher observed one of the two follow-up teacher classes for seven of the eight 
months in which one occurred. Although there were some technical difficulties in the 
observed classes, none prevented any school’s participation. Five of the seven observed 
classes included discussion, and the researcher rated the participants as “very engaged” in 
three meetings and “somewhat engaged” in two. 

Finally, a researcher observed seven of the nine administrator meetings. Technology 
difficulties prevented administrators from participating in one of the observed meetings. 

Due to the change in video conferencing software, the January 2005 meeting class was 
taught via phone and PowerPoint slides that were e-mailed to the participants. Since these 
meetings involved a small number of administrators as compared to the teacher meetings, 
they included more discussion between schools. The smaller number of participants gave 
individuals more opportunities to interact with administrators in the other schools. Except 
for the meeting that had technical problems, the observed administrator meetings 
included extensive on-topic discussion, and researchers rated the administrators as “very 
engaged.” 

Teacher Pre-post Intervention Survey 

Invitations to complete the pre-post intervention surveys were e-mailed to 123 teachers in the 
treatment group and 136 teachers in the control group. A total of 1 16 treatment teachers and 127 
control teachers completed the pre-intervention surveys, resulting in respective response rates of 
94 percent and 90 percent. A total of 93 teachers in the treatment group and 1 14 teachers in the 
control group completed the post-intervention surveys resulting in response rates of 76 percent 
and 84 percent respectively. Forty-five treatment teachers and 25 control teachers completed 
only the pre-intervention survey; 22 treatment teachers and 17 control teachers completed only 
the post- intervention survey. Nine survey respondents in the treatment group and 10 in the 
control group were not classroom teachers (e.g., librarian, counselor, paraeducator), and these 
persons were excluded from the two samples. After this adjustment, the number of teachers 
whose pre-intervention surveys could be matched with their post- intervention surveys was 7 1 in 
the treatment group and 97 in the control group. All data analyses reported for the pre-post 
intervention surveys are based on these samples. 

Table 6 shows the demographic characteristics of the teachers in the treatment and control 
groups as reported on the intervention survey. T-tests indicated statistically significant 
differences between the groups in years of teaching experience (p < .05) and years at current 
school (p < .05), with the control teachers reporting more teaching experience than the treatment 
teachers. The two groups of teachers did not statistically differ on other characteristics. 
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Table 6. Demographic Characteristics of Teachers in the Treatment and Control Groups 



Teacher Characteristic 


RTI Treatment Group 
n = 7I 


Control Group 
n = 97 


Grade level 


47% elementary 
39% secondary 
14% K-12* 


47% elementary 
41% secondary 
12% K-12* 


Mean years teaching 


14.26 years 


19.06 years 


Mean years at current school 


9.97 years 


12.92 years 


Masters degree 


25 % 


15 % 


Regular state certification 


96% 


98% 


Internet access in classroom 


93% 


96% 


Teaching is first career 


86% 


82% 



* Generally music, art, and physical education teachers. 



Six constructs on the pre-post intervention survey were measured by scales. The reliabilities for 
these scales on the pre- and post-surveys are indicated in Table 7. The Chronbach’s alphas 
ranged from .71 to .93, indicating their adequacy in measuring the targeted constructs. 



Table 7. Reliabilities of Scales on Teacher Pre-Post Intervention Survey 



Scale 


Chronbach’s alpha* 


Pre- 

survey 


Post- 

survey 


Professional community — shared norms and values (5 
items) 


.82 


.85 


Teacher efficacy (10 items) 


.71 


.75 


School environment — general support (9 items) 


.79 


.80 


School environment — principal support (9 items) 


.92 


.93 


RTI impact on teacher instruction (6 items; post- 
intervention survey only) 




.96 


Teacher satisfaction with the RTI (7 items; post-intervention 
survey only) 




.91 



* Measure of internal consistency of the scale items 



Teacher Participation in the Treatment (RTI) 

Teachers in the treatment group were asked about the extent of their participation in the RTI 
professional development program. The responses of the teachers in the matched sample were: 3 
percent “very limited extent,” 14 percent “some extent,” 17 percent “considerable extent,” and 7 
percent “great extent.” Eighty percent of these teachers reported that they attended “8-9” of the 
nine general meetings; 16 percent attended “5-7” general meetings, and four percent attended 
“2-4” general meetings. Sixty-three percent reported that they attended “8-9” of the nine follow- 
up meetings; 17 percent attended “5-7” follow-up meetings; and 20 percent attended “2-4” 
follow-up meetings.^ Teachers’ participation in the discussion boards varied more than their 



School records of teacher attendance at the RTI meetings were similar to the attendance reported by teachers. 
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participation in the class meetings. Forty-one percent of the treatment teachers reported that they 
read “8-9” of the nine discussions; 28 percent read “5-7” discussions, 23 percent read “2-4” 
discussions, and seven percent read “0-1” discussion. On average, the teachers reported they 
posted “2-4” messages to the discussion boards and “2-4” classroom materials to the RTI 
website. These data suggest that teachers participated more in the online classes than in the RTI 
discussions. 

The responses to each of the questions concerning level of participation were summed to obtain 
an overall measure of participation level. Forty-three treatment teachers were ranked as being at 
or above median participation (“high-participation RTI”), and 26 treatment teachers were ranked 
as below median participation (“low-participation RTI”).^ Table 8 show the percentages of the 
teachers in the high-RTI and low-RTI groups who reported participating in the different RTI 
activities and illustrates the differences in participation between the two groups. The data 
indicate that teachers varied in the amount of their exposure to the RTI intervention. For this 
reason, level of participation was used as a grouping variable in the analyses of teacher survey 
data. 



Table 8. Treatment Teachers’ Reported Participation in RTI Activities 



Type of RTI Activity 


Number of RTI Activities 


O-I* 


2-4 


5-7 


8-9 


Attended RTI general online meetings 










High participation** 






5% 


95% 


Low participation*** 




11% 


35% 


54% 


Attended RTI follow-up online meetings 










High participation 






9% 


91% 


Low participation 


31% 


23% 


31% 


15% 


Read online discussions? 










High participation 




5% 


32% 


63% 


Low participation 


19% 


54% 


23% 


4% 


Posted a message to online discussions 










High participation 


5% 


30% 


35% 


30% 


Low participation 


42% 


46% 


12% 




Posted a lesson plan or classroom materials to 










RTI website 










High participation 


33% 


37% 


23% 


7% 


Low participation 


73% 


27% 







*Refers to the number of meetings, discussions, or postings depending on the activity 
** n = 43 teachers 
*** n = 26 teachers 



Teachers had the opportunity to receive one hour of college credit at their own expense for their 
participation in the RTI. Forty-three percent of the treatment teachers reported they were 



^ Two treatment teachers did not answer questions about their participation in the RTI; therefore, the responses of 
these two teachers were not included in any analyses that examined influences of participation. 
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receiving college credit — 27 percent of the low-participation RTI teachers and 5 1 percent of the 
high-participation RTI teachers. There was a significant correlation between reported 
participation and receiving college credit (contingency coefficient = .23, p < .05) indicating that 
obtaining college credit was associated with a higher level of reported participation in the RTI 
(see Table 9). This finding suggests that receiving college credit motivated teachers to participate 
more than they might have without college credit. It also suggests that providing college credit 
for participation in the RTI (and other professional development) can be an incentive for 
teachers. 

Table 9. Number of Treatment Teachers Receiving College Credit for RTI Participation 





High RTI 
Participation 
n = 43 


Low RTI 
Participation 
n = 26 


College credit 


22 


7 


No college credit 


21 


19 



What is the Effect of the RTI on Teacher Professional Community? 

The intervention survey measured teacher professional community through the constructs of 
shared norms/values and teacher collaboration (see Table 5). There were no significant 
differences between the treatment and control groups in shared norms/values on the pre-or post- 
intervention surveys. The treatment and control groups did not differ significantly in the mean 
frequency of different types of collaboration on the pre-intervention survey. In an Analysis of 
Variance (ANOVA) of difference scores between the pre- and post-intervention surveys, 
significant differences were found among the high-participation RTI, low-participation RTI, and 
control groups’ pre-post change frequency of collaboration. As expected, there was a 
significantly greater increase in online collaboration for the treatment groups as compared to the 
control group (p < .001), and the increase was significantly larger for the high-participation RTI 
group than for the low -participation RTI group (p < .05). However, there were also significant 
differences in the mean change in frequency of collaborative activities that were not online ip < 
.05), such as peer observation and peer feedback. The control group reported a mean decrease in 
non-online collaboration, and the treatment groups reported a mean increase, which again was 
significantly larger for the teachers in the high-participation RTI group than the low-participation 
RTI teachers ip < .05). This pattern of findings is summarized in Table 10 and suggests that the 
RTI promoted both online and non-online collaboration, especially for those teachers with higher 
levels of participation. 

Online Discussions Content Analysis 

The purpose of the online teacher discussions in the RTI intervention is to encourage a high-level 
discussion of teaching practice and to create professional community within and across RTI 
schools. Teachers had approximately three weeks to participate in each of nine monthly 
discussions. A guide was developed to code the content, depth of discussion, and professional 
community observed in the discussions. The level of coding and analysis was at the message 
level, and because messages could contain multiple ideas, multiple codes were applied to each 
message. Two coders independently coded each message in the discussions and discussed any 
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discrepancies in their codes. In the few instances where they could not agree, a third coder 
resolved the discrepancies. 



Table 10. Mean Frequency of Teacher Collaboration 



RTI 

Participation 


Pre-intervention Survey 


Post-intervention Survey 


Online Teacher 
Collaboration 


Non-online 

Teacher 

Collaboration 


Online Teacher 
Collaboration 


Non-online 

Teacher 

Collaboration 


Mean SD 


Mean SD 


Mean SD 


Mean SD 


High n = 43 


1.26 .59 


2.21 .60 


2.30 .80 


2.45 .64 


Eow n = 26 


1.27 .45 


2.25 .50 


1.88 .83 


2.41 .75 


None (control) 
n = 97 


1.36 .78 


2.42 .61 


1.40 .85 


2.35 .61 



Response set: 1 = never, 2 = occasionally, 3 = monthly, 4 = weekly, 5 = almost daily 



Content coding was based on whether the message contained material that was relevant to 
general aspects of teaching, the RTI content, the use of technology in teaching, or topics 
unrelated to teaching. The coding for depth of discussion was adapted from the coding schemes 
used by Entwisle and Waterson (1988) and Henri (1992). Messages were coded at one of the 
following levels: (1) surface level processing — no new information added to ongoing 
discussion about teaching; (2) midlevel processing — adds an example to what has already been 
said but without any analysis of the example; (3) in-depth processing — adds new information 
and analysis to ongoing discussion about teaching. 

Each message was coded for four aspects of professional community: reflective dialogue on 
instructional practice, focus on student learning, deprivatization of practice, and collaboration 
(Eouis & Marks, 1998). A message was coded as containing reflective dialogue if it concerned 
specific issues of instructional practice. Examples include teachers’ discussion of problems and 
solutions for specific students or teaching practices that they had tried. A message was coded as 
containing a focus on student learning if it included explicit discussion of ways to promote 
student learning of content such as deeper understanding or higher-order thinking (as opposed to 
improving behavior or providing entertainment). Each message was coded for the following 
levels of deprivatization: (1) none; (2) moderate — shares general non-detailed examples of 
classroom practice; (3) high — shares classroom practice with a medium level of detail; (4) very 
high — shares details of student performance and/or lesson plans, materials, or curriculum they 
have used. A message was coded as collaboration if it included signs of teachers working 
together on class content, themes, curriculum, class outlines, lesson plans, class materials, 
projects, or team teaching. 

A sample of two of the nine threaded discussions was analyzed. These were the Homework and 
Practice discussion, which occurred in the fourth month of the intervention, and the discussion of 
Nonlinguistic Representation, which occurred in the six month. These discussions were chosen 
for analysis because teachers had sufficient time to become accustomed to the technology and 
they occurred at a time when teachers were not distracted with state testing of students. 

Eor the discussion of Homework and Practice, the discussion prompts provided by the instructor 
addressed the purposes, types, and effectiveness of homework/practice assignments. Eor the 
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discussion of Nonlinguistic Representation, the discussion prompts asked teachers to implement 
the strategy in a lesson with and without technology and then discuss their experiences and their 
students’ reactions. There were 103 messages posted to the Homework discussion, and 122 
messages posted to the Nonlinguistic discussion. Eighty-seven different teachers posted to the 
Homework discussion, and 67 different teachers posted to the Nonlinguistic discussion. 

Topic and content. All messages in the Homework discussion contained content relevant to 
teaching and to the RTI, and none contained content that was not on topic. Two messages 
contained content relevant to using technology in teaching. All messages in the Nonlinguistic 
discussion contained content relevant to teaching, and only one message was not relevant to the 
RTI. In addition, 33 messages (35%) contained content relevant to using technology in teaching 
(which was included in the prompt for the Nonlinguistic discussion). 

Depth of discussion. Just over half (54%) of the messages in the Homework discussion were 
coded as exhibiting in-depth processing, and an additional 29 percent exhibited midlevel 
processing through shared examples or experiences. Thirty-four percent of the messages posted 
to the Nonlinguistic discussion were coded as containing in-depth processing, and an additional 
45 percent were coded as midlevel processing. 

Professionai community. Ninety-nine percent of the messages in the Homework discussion 
and 85 percent of the messages in the Nonlinguistic discussion were coded as including 
reflective dialogue. Seventy-seven percent of the Homework messages included discussion of 
ways to promote student learning. The Nonlinguistic prompts tended to elicit examples of 
practice more than reflections on student learning; nonetheless, 40 percent of these messages 
were coded as including a focus on student learning. In the Homework discussion there was a 
moderate level of deprivitization of practice; 53 percent of the messages shared examples of the 
respondent’s classroom practice, but none shared detailed classroom practice or materials. There 
was stronger evidence of deprivitization of practice in the Nonlinguistic discussion; 95 percent of 
the messages showed some level of deprivitization. Forty-seven percent included examples of 
the respondent’s classroom practice, and an additional 25 percent shared detailed examples. 

None of the messages in the Homework discussion suggested teachers working together on class 
materials. There were three messages in the Nonlinguistic discussion in which teachers 
mentioned working with other teachers. These results for collaboration are not surprising given 
the nature of the discussion prompts, which did not ask for collaborative input regarding 
instruction or materials. Graphs of the coding results for professional community are contained 
in Appendix B. 

Online Discussions Survey Data 

An additional data source regarding the online discussion and the creation of professional 
community was the post- intervention survey. As shown in Table 11, teachers in the high- 
participation RTI group had numerically higher means than the teachers in the low-participation 
RTI group on the extent to which they perceived that the online discussions influenced their 
feeling of being connected with other teachers and created opportunities for peer mentoring. 



’ These numbers include treatment teachers for whom pre- and post-interventions surveys could not be matched. 
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However, t-tests indicated no significant differences between these two groups on either item 
(feeling connected: p < .06; mentoring opportunities: p < .12). 



Table 11. Treatment Teachers’ Perceptions of RTI Online Discussions 



Teacher Post-intervention 
Survey Item 


High RTI 
Participation 
n = 43 


Low RTI 
Participation 
n = 23 


Mean SD 


Mean SD 


Extent to which RTI online discussions 
made teachers feel connected with 
other teachers 


3.07 .86 


2.62 1.13 


Extent to which RTI online discussions 
created opportunities for peer 
mentoring 


2.95 1.02 


2.54 1.14 



Response set: 1 = not at all, 2 = very limited extent, 3 = some extent, 4 = considerable extent, 5 = great extent 



Summary of Professional Community Findings 

Results of the pre-post intervention surveys indicated that the RTI did not change teachers’ 
perceptions of shared norms and values but did increase the frequency of collaboration with 
other teachers — both online and non-online collaboration. The online discussions that were 
coded provided evidence for the creation of professional community, especially in reflective 
dialogue but also in the focus on student learning and deprivatization of practice. Data from the 
post-intervention survey suggested that teachers who participated more in the RTI may have felt 
slightly more connected with other teachers compared to treatment teachers who participated 
less. 

What is the Effect of the RTI on Variables Related to Teacher Retention? 

This section describes results related to teacher retention, including efficacy for teaching, job 
satisfaction, and intent to stay in teaching. Efficacy for teaching was measured by the teacher 
efficacy scale. The treatment and control groups did not significantly differ in mean teaching 
efficacy on the pre-intervention survey, the post-intervention survey, or in the analysis of pre- 
post difference scores. As Table 12 indicates, difference scores were in the predicted direction, 
with a mean decrease in efficacy for the control teachers and a mean increase for the treatment 
teachers, but these differences were not statistically significant {p < .41). 



Table 12. Teacher Efficacy on Pre- and Post- intervention Surveys 



RTI Participation 


Pre-intervention 

Survey 


Post-intervention 

Survey 


Mean SD 


Mean SD 


High n = 43 


2.83 .35 


2.85 .36 


Eow n = 26 


2.71 .36 


2.78 .35 


None (control) n = 97 


2.83 .35 


2.82 .40 



Response set: 1 = strongly disagree, 2 = somewhat disagree, 3 = somewhat agree, 4 = strongly agree 
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Job satisfaction and intent to stay in teaching were measured by four and two items respectively 
on the teacher pre-post intervention survey. Because these items addressed similar concepts, a 
Multivariate Analysis of Variance (MANOVA) was conducted on the scores for the six items on 
the pre-intervention surveys and on the pre-post difference scores. The treatment and control 
groups did not significantly differ on any of these items on the pre-intervention survey or in the 
analysis of pre-post difference scores. Although not statistically significant (p < .14), the largest 
numerical increase between pre- and post-intervention surveys occurred for the low participation 
RTI group in their intent to remain in teaching as long as they are working (see table 13). 
Overall, the results indicate medium to high levels of satisfaction with teaching except for the 
salaries, with which teachers were less satisfied. 

A question about intention to return to current teaching assignment was asked on the post- 
intervention survey only. Although an ANOVA indicated that the differences were not 
statistically significant (p < .19), as indicated in Table 13, the teachers in the low-participation 
RTI group had the lowest mean on this measure. 



Table 13. Teachers’ Mean Responses Concerning Job Satisfaction and Intention to Stay in 
Teaching 





High RTI 


Low RTI 


No RTI 




Participation 
n = 43 


Participation 
n = 26 


Participation 

(control) 


Teacher Post-intervention 










n= 


97 


Survey Item 


Pre- 


Post- 


Pre- 


Post- 


Pre- 


Post- 




survey 


survey 


survey 


survey 


survey 


survey 




Mean 


Mean 


Mean 


Mean 


Mean 


Mean 




SD 


SD 


SD 


SD 


SD 


SD 


If I could go back to my college 


3.56 


3.51 


3.15 


3.27 


3.32 


3.29 


days and start over again, I would 
still be a teacher 


.67 


.70 


.92 


.78 


.84 


.82 


All things considered, I am 


3.56 


3.58 


3.46 


3.38 


3.52 


3.51 


satisfied with being a teacher 


.50 


.54 


.65 


.57 


.63 


.61 


I am satisfied with the working 


3.16 


3.19 


3.31 


3.31 


3.21 


3.02 


environment at this school 


.65 


.70 


.62 


.73 


.78 


.85 


I am satisfied with my teaching 


2.16 


2.28 


2.58 


2.46 


2.25 


2.23 


salary 


.97 


.85 


.90 


1.10 


.92 


.88 


At some point, I will probably try 


2.10 


2.23 


2.46 


2.35 


1.96 


2.11 


to find a teaching job in another 
community 


.93 


.92 


.90 


1.02 


1.01 


1.00 


I plan to remain in teaching or K- 


3.40 


3.48 


2.85 


3.08 


3.42 


3.68 


12 education as long as I’m 
working 


.76 


86 


1.12 


.80 


.79 


.69 


I plan to return to this school in 




3.77 




3.46 




3.68 


2005-2006 




.48 




.86 




.48 



Response set: 1 = strongly disagree, 2 = somewhat disagree, 3 = somewhat agree, 4 = strongly agree 



A final set of 17 survey items related to job satisfaction asked teachers about the degree of 
influence that various factors had on their thinking about whether to continue teaching. 
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Univariate ANOVAs were eondueted on responses to these items on the pre- and post- 
intervention surveys and on the pre -post differenee seores. The only signifieant differenee was in 
the faetor of “professional development opportunities at this sehool” ip < .05). The high- 
partieipation RTI group deereased signifieantly in their pereeptions of professional development 
as an important faetor from the pre- to post-intervention surveys, while the low-partieipation RTI 
group and the eontrol group inereased in their pereeptions of this as an important faetor to 
eonsider in deeiding whether to eontinue teaehing. A similar but non-signifieant pattern was 
observed for the faetor of “other teaehers in this sehool” ip < .10). The overall means and 
standard deviations for the influenee of the 17 different faetors are given in Appendix B. The 
faetors with mean ratings of 3.5 and above (on a 4-point seale where 4 = “strong influenee”) 
were “working with the students,” “the rural lifestyle/eommunity,” and “elose to family and/or 
home.” The faetors with mean ratings of 2.5 or lower were “aeeess to reereational opportunities 
nearby” and “having no viable job alternatives to teaehing in this geographieal area.” 

Summary of Teacher Retention Findings 

In general, the findings indieate that the RTI did not have statistieally signifieant effeets on the 
variables related to teaeher retention in this study. There was some suggestion that results were 
in the desired direetion (e.g., for teaeher effieaey), so it is possible that differenees would be 
deteeted with a larger sample size. The analyses of faetors that teaehers pereeived as influeneing 
their deeision to eontinue teaehing produeed an unexpeeted pattern. The high-partieipation RTI 
group pereeived professional development and the other teaehers in the sehool as less important 
influenees after the RTI intervention while the opposite oeeurred for the low-partieipation RTI 
and eontrol groups. Perhaps the high-partieipation RTI treatment teaehers pereeived less need for 
these elements after the RTI oeeurred. Overall, the faetors with the strongest reported influenee 
on whether teaehers in the study eontinue to teaeh eoneemed aspeets of both the profession 
(working with ehildren) and the loeation (rural and elose to family). 

What is the Influence of the RTI on Teachers’ Use of Effective 
Instructional Practices? 

This researeh question eoneems teaehers’ use of the nine eategories of instruetional strategies 
from Classroom Instruction That Works (Marzano et ah, 2001) that RTI teaehers were taught in 
the monthly online elasses. The primary souree of data related to this question is the post- 
intervention survey of teaehers in the RTI treatment group. The teaehers were asked to rate on a 
5-point seale how mueh they ehanged their frequeney of use of eight different instruetional 
strategies, from “mueh less use” to “mueh more use,” eompared to their instruetion prior to the 
RTI. (The post-survey was administered to teaehers prior to the last elass on the ninth strategy of 
eooperative learning, so a question eoneerning that strategy — the ninth — was not on the post- 
survey.) The mean response for teaehers’ reported ehange in the use of the strategies was 3.52 
for high-partieipation RTI teaehers (“more use”) and 3.20 for low-partieipation RTI teaehers 
(“about the same use”). The t-test of the differenee between the partieipation groups was 
statistieally signifieant ip < .001). A MANOVA resulted in statistieally signifieant differenees 
favoring the high-partieipation RTI group on four of the eight strategies, as indieated in Figure I. 
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□ High RTI (n=43) □ Low RTI (n=26) 

Ui 




Instructional Strategies 

Scale: 1=much less use, 2=less use, 3=about the same use, 

4=more use, 5=much more use 
(*p < .05) 

Figure 1. Treatment Teachers’ Reported Use of Instructional Strategies Compared to 
Their Instruction Prior to the RTI 

RTI teachers also were asked three questions related to the frequency with which they 
implemented RTI instructional and technology strategies. Table 14 shows the means and 
standard deviations for these questions. Based on t-tests, teachers in the high-participation RTI 
group reported significantly more frequent use of RTI instructional or technology strategies in 
lessons (“monthly”) than teachers in the low-participation RTI group (“occasionally”) (p < .01). 
High RTI teachers reported more frequent discussion related to the strategies, but the difference 
was not statistically significant (both “occasionally”), and the two groups reported the same 
frequency (“occasionally”) of working with other teachers on using the strategies. 

Another source of data concerning teachers’ use of RTI strategies is a set of four questions on the 
administrator surveys given after the RTI ended in May 2005. Administrators were asked the 
approximate number of times they observed most of their teachers in their classroom since the 
beginning of the school year. The average number of observations was between two to three 
times, and the administrators in the two groups did not significantly differ. As indicated in Table 
15, the RTI administrators perceived increased use of RTI instructional strategies in more than 
50 percent of their teachers, while control administrators perceived increased use of different 
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instructional strategies in 26-50 percent of their teachers, a difference that was statistically 
significant ip < .05). RTI administrators also perceived that more of their teachers improved and 
more increased their use of technology compared to the perceptions of control administrators, but 
these differences were not statistically significant. 



Table 14. Treatment Teachers’ Reported Frequency of Implementing RTI Strategies 



Treatment Teacher 
Post-intervention Survey 


High RTI 
Participation 
n = 43 


Low RTI 
Participation 
n = 26 




Mean 


SD 


Mean 


SD 


Use RTI instructional or technology 
strategies in a lesson. 


3.14 


.92 


2.50 


.91 


Work with other teachers (online or in 
person) to develop classroom materials or 
lesson plans that incorporate RTI 
instructional or technology strategies. 


1.98 


.86 


1.92 


.80 


Discuss with other teachers (online or in 
person) a lesson or classroom materials that 
use RTI instructional or technology 
strategies. 


2.51 


.80 


2.27 


.87 


Response set: 1 = never, 2 = occasionally (1-8 times year), 3 = monthly (1-3 times a month), 4 
a week), 5 = almost daily (more than 3 times a week) 


= weekly ( 1 - 


Table 15. Treatment Administrators’ Perceptions of Teaching 






Administrator Survey Item 

In the past year, what percentage of 


Treatment 

Administrators 

n=I0 


Control 
Administrators 
n = I2 


teachers in your school would you 
say...? 


Mean 


SD 


Mean 


SD 


Improved their overall teaching quality 


3.40 


.70 


2.92 


.90 


Increased their use of technology in their 
classrooms 


3.10 


.60 


2.75 


.87 


Increased their use of RTI instructional 
strategies (treatment) 

Increased their use of different 
instructional strategies (control) 


3.60 


1.08 


2.64 


.81 



Response set; 1 = none, 2 = l%-25%, 3 = 26%-50%, 4 = 51%-75%, 5 = 76%-100% 



Summary of Findings on Treatment Teachers’ Instructional Practices. 

Teachers who reported higher levels of participation in the RTI reported a greater increase in use 
of the instructional strategies in Classroom Instruction That Works (Marzano et ah, 2001) 
compared to teachers who reported lower levels of participation in the RTI. The high- 
participation RTI teachers also indicated more frequent use of RTI instructional or technology 
strategies in lessons as compared to the low-participation RTI teachers. However, because the 
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questions about strategy use were asked on the post-survey only, caution is warranted in 
interpreting the results as indicating teacher change. 



The administrators of RTI schools perceived increased use of RTI instructional strategies by over 
50 percent of the RTI teachers. There was a trend in the data suggesting that RTI administrators 
perceived more improvement in teaching quality over the past school year as compared to 
administrators in control schools. This difference was not statistically significant (p < .18) 
possibly due to the small sample size of the administrators. However, the result does suggest that 
RTI administrators perceived positive changes in the instruction of many of their teachers. 

How Effective Is the RTI at Increasing Teacher and Administrator 
Comfort with and Use of Technology? 

The primary data source for this question is the pre-post teacher technology survey. This survey 
included scales designed to measure teachers’ comfort with technology, their perceived 
technology skills, and their perceptions of the school’s acceptance of technology. The 
reliabilities for these scales are shown in Table 16. The Chronbach's alphas ranged from .65 to 
.92, indicating their adequacy in measuring the targeted constructs 



Table 16. Reliabilities of Scales on Teacher Pre-Post Technology Survey 



Scale 


Chronbach’s alpha 


Pre- 

survey 


Post- 

survey 


Teacher comfort with technology (11 items) 
Response: 1 = very comfortable, 5 = very uncomfortable 


.91 


.88 


Teacher technology skill (10 items) 
Response: 1 = no skill at all, 5 = expert. 


.92 


.92 


School technology acceptance (9 items) 
Response: 1 = strongly agree, 5 = strongly disagree 


.65 


.75 



Invitations to complete the pre-post technology surveys were sent to 123 teachers in the 
treatment group and 136 teachers in the control group. There were 109 teachers in the treatment 
group and 127 teachers in the control group who completed the pre-technology survey, for 
response rates of 87 percent and 93 percent respectively. Seventy-nine treatment teachers and 
112 control teachers completed the post-technology surveys, for response rates of 64 percent and 
82 percent respectively. The technology pre-survey was conducted via paper and pencil, and the 
technology post-survey was conducted online, which created difficulties in matching teachers’ 
pre-post surveys. As a result, surveys were matched for 26 RTI (treatment) teachers and 35 
control teachers."^ On the pre-survey, the 109 RTI teachers who responded had significantly 
higher perceptions of technology skill and school acceptance of technology than the 112 control 
teachers. Thus, the two groups were not equivalent in technology perceptions prior to the RTI. 
For this reason, the matched sample of 61 teachers who responded to the technology surveys was 



In contrast to the sample of teachers who responded to the pre-post intervention survey, respondents who were not 
classroom teachers were retained in the sample for the teacher technology survey because the latter targeted 
technology-related change more than impact on instruction. 
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analyzed despite its relatively small size. Difference scores were created to measure the amount 
of change in these teachers’ means on the three scales in Table 15, and separate ANOVAs were 
conducted. 

Among the matched sample of teachers, the level of comfort with technology was not 
significantly different between the RTI and control groups on either the pre-survey or post- 
survey. All teachers increased in technology comfort from pre to post ip < .01), but the amount 
of change was statistically similar for the two groups (see Table 17). The level of teachers’ 
perceived skill with technology was significantly higher among the teachers in the treatment 
group than in the control group on both the pre- survey (p < .01) and post- survey (p < .05). The 
mean pre-post difference scores in technology skill were not significantly different between the 
RTI and control teachers, although there was a weak trend towards an increase for both groups. 
The perceptions of the treatment teachers with regard to their school’s acceptance of technology 
were significantly higher than those of the control teachers on both the pre-survey (p < .01) and 
post-survey (p < .05), with no significant differences in the analysis of pre-post change. 



Table 17. Teachers’ Perceptions of Technology Comfort, Skill, and School Acceptance 



Teacher Pre-post 
Technology Survey 


RTI Treatment 
n = 26 


Control 
n = 35 


Pre- 

survey 


Post- survey 


Pre- 

survey 


Post- 

survey 


Mean SD 


Mean SD 


Mean SD 


Mean SD 


Teacher comfort with technology 










Response set; 1 = strongly agree, 


3.71 1.08 


4.16 .82 


3.26 


4.04 .79 


5 = strongly disagree 






1.08 




Teacher technology skill 










Response set: 1 = no skill at all, 


3.39 .68 


3.69 .58 


3.10 


3.27 .65 


5 = expert 






.75 




Technology environment 










Response set: 1 = very comfortable, 


3.81 .53 


3.86 .51 


3.56 


3.50 .45 


5 = very uncomfortable 






.45 





The sample of teachers who responded to the post-technology survey was examined to determine 
if differences in their level of participation in the RTI were related to their perceptions. 

Univariate ANOVAs revealed no significant differences among the high RTI, low RTI, and 
control groups for any of the three constructs. Differences among teachers’ perceived technology 
skill on the post-technology survey approached significance (p < .06), with higher perceived 
technology skill by teachers with higher levels of participation (M = 3.16, 3.23, and 3.56 for 
control, low-participation RTI, and high-participation RTI teachers respectively). 

The pre -post technology surveys administered to both RTI and control administrators provide a 
second source of data pertaining to the RTFs impact on perceptions related to technology. 
Response rates on the pre -post technology surveys for administrators in the RTI group were 
13/14 (93%) and 12/14 (86%) respectively. For administrators in the control group, response 
rates were 14/17 (82%) and 13/17 (76%) respectively. The technology pre-survey was conducted 
via paper and pencil, and the technology post-survey was conducted online, which created 
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difficulties in matching administrators’ pre-post surveys. Pre-surveys were matched to post- 
surveys for nine RTI and six control administrators. This matched pre-post administrator sample 
is used for analyses of questions that appeared on both the pre- and post technology surveys. 
Additional questions were asked on the post-survey only, and for these questions the entire post- 
survey administrator sample was used. Although significance testing is reported, due to the small 
sample sizes, administrator survey data will be discussed in terms of trends as well. 

Administrators were asked on the pre-post technology surveys how supportive the community, 
district, and faculty were of using technology in the school. There were no significant differences 
between administrators on either the pre- or post- levels of support, nor were there any significant 
differences in the pre-post difference scores between the RTI and control group. This suggests 
that change in community, district, or faculty support during the RTI was not a factor in the 
study findings regarding technology integration. In all schools, administrators perceived support 
for technology integration. (See Appendix B, Table B2.) 

Administrators were asked only on the post-survey what percentage of their teachers increased 
their use of technology in their classrooms over the past year. Answers were given in ranges (1 = 
none, 2 = 1-25% , 3 = 26-50%, 4 = 51-75%, 5 = 76-100%). There was a non-significant trend 
for RTI administrators to report a higher average percentage range (M = 3. 10, SZ) = .74) than 
control administrators (M = 2.75, SD = .87). 

The post- technology survey also asked both groups of administrators 12 questions related to 
technology environment and attitudes toward technology. The means and standard deviations for 
seven of the items are shown in Table 18. The remaining five items concern leadership and are 
discussed in the next section. Based on a MANOVA, the RTI administrators had significantly 
more agreement than the control administrators with statements concerning beliefs about 
computers stimulating creativity and teachers’ knowledge about integrating technology in the 
classroom (p < .05). However, because there are no pre-survey data for these items, the group 
differences might have existed prior to the RTI intervention. 

Summary of Technology Findings 

Findings based on the matched pre-post teacher technology sample indicate that the RTI did not 
have statistically significant effects on increasing teachers’ comfort with technology compared 
with the control sample. Teachers in both the RTI and control groups experienced significant 
increases in comfort with technology. There was a trend toward increases in technology skill 
over the course of the study, but this did not differ significantly between the treatment and 
control teachers. 

Findings based on the sample of all the administrators who completed the post-technology 
survey suggest that RTI administrators perceived moderate positive effects on technology 
integration. There was a trend for RTI administrators to report a higher percentage of teachers 
using technology more in classrooms over the past year than control administrators. Given the 
small sample size of administrators involved in this study, it is possible that this difference would 
be statistically significant with a larger sample size. Further, RTI administrators were 
significantly more likely than control administrators to agree that their teachers know how to 
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integrate technology in their classroom to improve student achievement. However, the RTI 
teachers posted significantly higher perceptions of technology skill and school acceptance of 
technology on the pre-survey as compared to teachers in the control group, which could indicate 
pre-existing differences. 



Table 18. Administrators’ Mean Responses Concerning Technology Environment on the 
Post- Technology Survey 



Administrator Post-Technology Survey 


RTI 

Administrators 

n=12 


Control 

Administrators 

n=13 




Mean 


SD 


Mean 


SD 


The district encourages the use of technology in your 
school. 


3.50 


.52 


3.46 


.66 


The community is supportive of using technology in our 
school. 


3.17 


.39 


3.00 


.58 


Teachers in my school are in favor of using technology in 
their classrooms. 


3.08 


.29 


2.92 


.64 


Teachers in my school know how technology can be 
integrated into their classrooms to improve student 
achievement.* 


3.17 


.39 


2.62 


.51 


I believe that the use of computers in education reduces 
the personal interaction between teachers and students. 


3.17 


.58 


2.85 


.38 


Technology has been helpful in meeting district and state 
standards. 


3.00 


.60 


2.69 


.48 


Technology makes teaching more effective. 


3.25 


.62 


3.08 


.28 


I feel that computers are useful as instructional aids. 


3.33 


.49 


3.08 


.13 


I believe that computers can stimulate student creativity.* 


3.58 


.52 


3.15 


.38 



Response set: 1 = strongly disagree, 2 = somewhat disagree, 3 = somewhat agree, 4 = strongly agree 



* P < .05 

What Is The Influence of the RTI on Administrators’ Leadership Practice? 

The primary data for this question come from items on the teacher pre-post intervention survey 
that concerned school environment. As indicated in Tables 5 and 7, there were nine survey items 
that measured general aspects of school environment and seven that measured those aspects of 
school environment associated with principal support. The post-intervention teacher survey 
included an additional four items related to principal support that were not on the pre- 
intervention survey (see Table 19). 

ANOVAs revealed no significant differences between teachers in the RTI and control groups on 
the school environment and principal support scales on either the pre- or post-intervention 
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survey. Nor were there any significant differences between these groups of teachers on the four 
additional post-survey items. ANOVAs also showed no significant differences on the two school 
environment scales among high RTI, low RTI, and control groups on either the pre- or post- 
intervention survey scales (see Table 19). Difference scores reflecting pre-post change were 
calculated for both scales. There was a slight increase in the mean perceptions of high- 
participation RTI teachers regarding school environment and principal support and a slight 
decrease in the perceptions of low-participation RTI teachers and control teachers; however, 
these changes were not statistically significant for the two groups. 



Table 19. Teachers’ Mean Perceptions of School Environment and Principal Support 



Teacher Pre-post 
Intervention 
Survey 


High RTI 
Participation 


Low RTI 
Participation 


No RTI 
Participation 


Pre- 

survey 


Post- 

survey 


Pre- 

survey 


Post- 

survey 


Pre- 

survey 


Post- 

survey 


Mean 

SD 


Mean 

SD 


Mean 

SD 


Mean 

SD 


Mean 

SD 


Mean 

SD 


School environment- general 


2.67 


2.72 


2.88 


2.88 


2.76 


2.71 


(9 items) 


.55 


.49 


.51 


.50 


.44 


.46 


School environment- principal 


2.90 


2.90 


3.16 


3.15 


2.92 


2.93 


support (9 items) 


.64 


.70 


.61 


.54 


.67 


.69 


I feel supported by my principal.* 




3.26 


— 


3.46 


— 


3.16 






.82 




.71 




.98 


The principal has made positive 




2.72 


— 


3.12 


-- 


2.93 


changes in his/her leadership in the 




.88 




.77 




.93 


past year.* 














The principal is supportive of 


— 


3.49 


— 


3.52 


— 


3.45 


integrating technology in the 




.59 




.65 




.65 


classroom* 














The principal is supportive of 


— 


3.63 


— 


3.65 


— 


3.54 


teacher participation in (RTI) 




.54 




.49 




.61 


professional development* 















* Single item, post-survey only 

Response set: 1 = strongly disagree, 2 = somewhat disagree, 3 = somewhat agree, 4 = strongly agree 



A second source of data concerning administrators’ leadership practice was the administrator 
post-technology survey discussed in the previous section. The post-survey included five items 
that were not on the pre-survey and that addressed administrators’ leadership practice with 
regard to technology (see Table 20.). The five survey items were clustered to form a technology 
leadership scale (Chronbach’s alpha = .62). As Table 20 indicates, the RTI administrators 
reported a significantly higher mean for technology leadership than the control administrators, a 
difference that approached significance ip < .07). Based on a MANOVA, administrators 
participating in the RTI showed significantly more agreement than the control administrators 
with respect to knowing how to integrate technology for the improvement of student 
achievement ip < .05) and helping teachers acquire technology for classroom projects (p < .01). 
However, there are no pre-survey data for these items, so group differences might have existed 
prior to the RTI intervention. 
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Table 20. Administrators’ Perceptions Concerning Technology Leadership 



Administrator Technology Post-survey 


RTI 

Administrators 

n=12 


Control 

Administrators 

n=13 




Mean 


SD 


Mean 


SD 


I encourage staff to use technology. 


3.58 


.52 


3.46 


.52 


I know how technology can be integrated into the classroom to 
improve student achievement.* 


3.42 


.52 


2.69 


.86 


Integration of technology into classrooms is a high priority for 
me. 


3.33 


.49 


3.15 


.37 


I use incentives to encourage faculty to participate in technology 
professional development. 


2.42 


.51 


2.38 


.77 


I help teachers acquire technology for their classroom 
projects.** 


3.42 


.52 


2.92 


.28 


Technology Leadership Overall Mean 


3.23 


.34 


2.98 


.31 



Response set; 1 = strongly disagree, 2 = somewhat disagree, 3 = somewhat agree, 4 = strongly agree 



* P < .05 
**P< .01 

Summary of Leadership Findings 

Findings based on the matched teacher sample indicate that the RTI did not significantly 
influence teachers’ perceptions of their principal’s leadership or school environment relative to 
the control group’s perceptions. Findings based on the sample of the administrators who 
completed the post-technology survey indicate higher levels of technology leadership for RTI 
administrators than control administrators, particularly for awareness of how to integrate 
technology and for assistance provided to teachers for technology integration. 

What Aspects of the RTI Intervention Did Participants Find Effective and 
What Improvements Should Be Made? 

The purpose of these questions is to examine participants’ reactions to the RTI and to solicit 
recommendations. Several sources of quantitative and qualitative data were used to answer these 
questions, and findings are organized according to the data source. 

Post-intervention Teacher Survey 

RTI teachers were asked to indicate the extent of their agreement with seven survey items about 
their satisfaction with the RTI professional development intervention. The reliability for the 
overall scale was high (Chronbach’s alpha = .91). The mean response of teachers was 2.73 on a 
4-point scale, indicating their general satisfaction with the RTI. Teacher satisfaction was 
significantly greater for high-participation RTI teachers than low-participation RTI teachers (p < 
.01) with means of 2.91 and 2.42, respectively (SD = .47 and .75). As shown in Figure 2, the 
high-participation RTI group agreed significantly more than the low-participation RTI group that 
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the online class meetings were interesting, that the content was useful and that the discussion 
groups were interesting and informative. 

□ High RTI (n=30) DLow RTI (n=17) 



c 




Treatment Teachers' Satisfaction with the RTi 

Scale:1=strongly disagree, 2=disagree, 3=agree, 4=strongly agree 

(**p < .01) 



Figure 2. Treatment Teachers’ Reported Satisfaction with the RTI 

Teachers in the treatment group also were asked six questions about the extent to which the RTI 
had improved aspects of their teaching and their students’ learning. The reliability for the scale 
formed from these items was .96 (Chronbach’s alpha). The teachers had a mean response of 2.92 
{SD = .81) or “some extent.” On average, perceived impacts were significantly higher for the 
high-participation RTI group than for the low-participation RTI group (p < .01). As shown in 
Figure 3, the RTI teachers with a high level of participation gave the intervention significantly 
higher impact ratings for six aspects of teaching and student learning than the RTI teachers with 
a low level of participation. 

Finally, teachers in the treatment group were asked how often technical difficulties significantly 
interfered with their school’s ability to connect with, hear, or see the RTI general or follow-up 
teacher online class. Forty-one percent of the teachers reported that technical difficulties “rarely” 
interfered (0-2 meetings); 37 percent said that they “infrequently” interfered (3-8 meetings); 17 
percent said they “frequently” interfered (9-14 meetings), and 4 percent said that technical 
difficulties “almost always” interfered (15-18 meetings). Eighteen of the 69 treatment teachers 
who responded to this question provided comments regarding these difficulties. Teachers 
indicated that they found these technical difficulties frustrating and not conducive to learning. 



The McREL Rural Technology Initiative: 
Research and Evaluation Study 



Page 32 



□ High RTI (n=43) HLow RTI (n=26) 
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RTI Impact on Teaching and Student Learning 

Scale: 1=not at all, 2=very limited extent, 3=some extent, 
4=considerable extent, 5=great extent 
(*p< .05, **p< .01) 



Figure 3. Treatment Teachers’ Perceived Impact of the RTI on Teaching and Student 
Learning 

Teacher Technology Survey 

Additional quantitative teacher data on the perceived impact of the RTI on teaching and learning 
come from the teacher post-technology survey discussed previously. Teachers in the treatment 
group were asked the extent of their agreement with seven statements about the RTFs impact on 
teaching and technology integration (see Table 21). Mean responses for all statements fell 
between “neither agree nor disagree” and “agree.” The high-participation RTI group had higher 
mean responses than the low-participation RTI group on five of the seven items. The differences 
were not statistically significant, possibly due to the smaller sample of teachers who completed 
the post-technology survey compared to the post-intervention survey^. The comparison between 
high and low participation approached significance for the statement “The McREL training has 
helped me think deeply about how I use technology in the classroom” (p < .06.). 



^ Using the participation measure from the post-intervention survey reduced the sample size for the post-technology 
because participation data were not available for all the teachers who took the post-technology survey. 
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Table 21. Treatment Teachers’ Perceptions of RTI Teaching and Learning Benefits 



Teacher Post- Technology Survey 


High RTI 
Participation 
Post- survey 
n = 30 


Low RTI 
Participation 
Post- survey 
n=I7 




Mean 


SD 


Mean 


SD 


The McREL training has helped me think deeply about 
how I use technology in the classroom. 


3.70 


.65 


3.24 


1.20 


My classroom has benefited as a result of tbe McREL 
training. 


3.73 


.65 


3.41 


1.50 


Teachers in my school share what they learn through the 
McREL training. 


3.47 


.86 


3.25 


1.07 


The McREL training has helped to improve the quality of 
classroom instruction in my school. 


3.70 


.60 


3.55 


1.18 


I receive adequate support from my school/district as I try 
to implement what I learn through the McREL training. 


3.77 


.82 


3.76 


1.09 


As a result of the McREL training, teachers in my school 
are more proficient in the use of technology. 


3.43 


.94 


3.47 


1.18 


As a result of the McREL training, students are using 
technology more in their learning. 


3.50 


.86 


3.41 


1.18 



Response set: 1 strongly disagree, 2 =disagree, 3 = neither agree nor disagree, 4= agree, 5 = strongly agree 



Administrator Satisfaction Survey 

The 13 RTI administrators were sent e-mail invitations to complete a satisfaction survey at the 
end of the RTI intervention. The survey asked about their school and personal participation in 
RTI meetings and discussions, perceptions of the RTI and its usefulness, and perceived effects of 
the RTI. Ten of the 13 RTI administrators completed the satisfaction survey (response rate = 
77%). Two of these respondents reported that their primary role in the school was technology 
coordinator, and the other eight reported that they were administrators. 

All ten of the respondents reported that their schools attended nearly all (8-9) of the nine general 
online class meetings that had occurred by the time that the survey was administered. Their 
reported school attendance for the follow-up teacher meetings was almost as high — nine of the 
ten administrators reported that their school attended 8-9 meetings. The administrators also were 
asked to what extent they personally participated in the RTI professional development program. 
They responded with a mean answer of 4.0 on a 5-point scale indicating a “considerable extent.” 
Four administrators reported that they personally attended “8-9” general teacher meetings, five 
attended “5-7” meetings, and one attended “2-4” meetings. Four administrators reported reading 
“8-9”of the nine threaded discussions, four read “2-4” discussions, and two read “0-1.” E- 
campus records showed that half of the administrators never posted to (i.e., contributed to) an 
online discussion, while one-third posted only once. Only two administrators posted regularly 
(7-8 posts each). Three administrators reported that they attended all six administrator online 
meetings; six attended “4-5” meetings; and one attended “2-3” meetings. 
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Administrators were asked to rate the quality and usefulness of different RTI components on a 5- 
point scale. As shown in Table 22, administrators’ quality and usefulness ratings on five of the 
seven components of the intervention were 4.00 or above, indicating administrators’ overall 
satisfaction with the RTI. 



Table 22. Treatment Administrators’ Perceptions of the Quality and Usefulness of RTI 
Components 



Administrator Satisfaction Survey 


Quality 
n = 10 


Usefulness 
n = 10 


Mean 


SD 


Mean 


SD 


The materials and resources provided 


4.40 


.97 


4.20 


.92 


The RTI technology (its operation and platform) 


3.20 


1.0 


3.50 


1.4 


The teacher e-mail discussion boards 


4.10 


.74 


4.00 


.94 


The general and follow-up teacher meetings 


4.00 


.82 


3.80 


.79 


The content presented in the teacher meetings 


4.50 


.97 


4.40 


.70 


The administrator sessions 


4.40 


.97 


4.56 


.53 


The content presented in the administrator meetings 


4.80 


.42 


4.80 


.42 



Response set: 1 = low, 5 = high 



Regarding technical difficulties, RTI administrators reported that connectivity was rarely a 
problem. They also indicated that they were rarely affected by other technical difficulties. 
Midway through the intervention, it was necessary to change the video conferencing software 
from WebEx to Marratech. Seventy percent of the administrators reported that the Marratech 
system worked better than the WebEx system. 

RTI administrators also were asked about the extent of the RTFs impact on their leadership. 
These questions did not have high reliability as a scale (Chronbach’s alpha = .48), and so are 
reported individually. As shown in Table 23, administrators reported that the RTI had “some” to 
“considerable” impact on various aspects of their leadership. The highest mean ratings of impact 
were for overall leadership, focusing on school priorities, and clarifying priorities as a school 
leader. An additional four questions concerned the influence of specific elements of the RTI on 
leadership. As Table 24 indicates, mean responses again ranged from “some” extent to 
“considerable” extent, with the highest mean reported for influence on organizational vision and 
priorities. 
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Table 23. RTI Administrators’ Perceptions of RTI Impact on Aspects of Leadership 



Administrator Satisfaction Survey 


RTI 

Administrators 
n = I0 




Mean 


SD 


Make you feel more connected with other administrators in your 
position 


2.63 


.92 


Allow you to give and get ideas for problems administrators in 
your position face 


3.38 


.74 


Improve your relationship with the staff in your school 


3.63 


.52 


Improve your ability to evaluate your staff 


3.63 


.52 


Improve your focus on which priorities are the most important 
for your school now and in the near future 


4.13 


.84 


Improve your overall leadership in your school 


4.00 


.00 


Clarify your priorities as a school leader 


4.00 


.76 


Observing the teachers RTI sessions has been useful to me as an 
administrator. 


3.80 


1.23 


The RTI work has begun to clarify my role in setting the stage 
for technology integration. 


3.80 


1.14 


The administrator RTI sessions have influenced my 
organizational vision. 


4.10 


.88 


I have developed a sense of community and shared purpose with 
the other administrators participating in the RTI. 


3.00 


1.16 



Response set: 1 = not at all, 2 = very limited extent 3 = some extent, 4 = considerable extent, 
5 = great extent, or NA7 1 don’t do this in my position 



Table 24. RTI Administrators’ Perceptions of RTI Influence 



Administrator Satisfaction Survey 


RTI 

Administrators 
n = I0 




Mean 


SD 


Observing the teachers’ RTI sessions has been useful to me as 
an administrator 


3.80 


1.23 


The RTI work has begun to clarify my role in setting the stage 
for technology integration. 


3.80 


1.14 


The administrator RTI sessions have influenced my 
organizational vision. 


4.10 


.88 


I have developed a sense of community and shared purpose with 
the other administrators participating in the RTI. 


3.00 


1.16 



Response set: 1 = not at all, 2 = very limited extent 3 = some extent, 4 = considerable extent, 
5 = great extent, or NA7 1 don’t do this in my position 
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RTI Administrator Interviews 



In October 2004, the 12 administrators and two teehnology eoordinators who were part of the 
RTI treatment group at that time were interviewed by telephone for the purpose of gathering 
formative information about the RTI. (One sehool later left the treatment group.) Data pertaining 
to intervention effeetiveness and improvements are reported here. All 12 administrators were 
positive about the administrator online meetings and the eontent eovered. Most reported enjoying 
the opportunity to talk with other administrators during these meetings. Eight of the 
administrators said that aceess to online professional development is “very important” because 
their sehools are more than an hour away from the nearest eollege or university that eould offer 
professional development. The remaining six administrators said that online delivery was 
“somewhat important” to them and referred to advantages in time and eonvenienee. 

Four administrators reported overhearing informal diseussions among their staff about the RTI 
instructional strategies, and eight administrators reported seeing some inerease in teaehers’ use 
of these strategies in the elassroom. There was also a pereeption of inereased teehnology eomfort 
and use in the elassroom among the six administrators in sehools that did not already foeus on 
teehnology. The other five said their teaehers were eomfortable with teehnology prior to the RTI, 
ereating a eeiling effeet in these sehools. Nine administrators reported positive ehange in sehool 
elimate sinee the RTI began. 

Administrators viewed the threaded online teaeher diseussion as helpful to those teaehers who 
were using it frequently, eommenting that the diseussion was the only way that their teaehers 
interaet with teaehers in other sehools. A suggestion was to replaee the one large diseussion with 
several different diseussions depending on the teaeher’ s grade level taught, and in the upper 
levels, the subjeet area. Another suggestion was to pair teaehers for e-mail eorrespondenee. 
Overall, administrators viewed the eontent of the RTI teaeher professional development 
{Classroom Instruction That Works, Marzano et al, 2001) as valuable and would not ehange it. 
Eleven administrators said teaeher partieipation in the RTI helped teaehers who were not already 
“highly qualified” by NCEB standards to go in the right direetion toward meeting these 
standards. 

Although the majority of administrators reported that their teaehers were engaged in the RTI 
professional development, some voieed eoneerns about individual teaehers who were less 
engaged and passive in the projeet. Twelve of the administrators said that offering eollege eredit 
for partieipation was very important and inereased teaeher motivation. Several administrators 
eommented on the need to inerease teaeher aeeountability for partieipation, espeeially for those 
who were reeeiving eollege eredit. Suggestions ineluded assigning prior reading, having more 
within-sehool diseussion during and after meetings, asking a group of teaehers in eaeh sehool to 
try the instruetional strategies and briefly report on their experienees in the next meeting or in the 
threaded diseussion. Teaehers in speeifie sehools also might be asked to share their experienees 
while other teaehers eould be asked to post lesson plans or assignments to the RTI website. 

Focus Groups 

After eompletion of the intervention, sehools were invited to send an administrator and teaeher to 
MeREE’s offiees in Aurora, Colorado, to partieipate in a elosing meeting. During this meeting. 
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separate focus groups of teachers and administrators were conducted to obtain feedback 
regarding the RTI, its influence on the staff at their schools, their concerns, and suggestions for 
improvement. Five teachers participated in the teacher focus group, four elementary and one 
secondary. The teachers represented five of the six treatment schools. Four administrators 
representing four of the treatment schools (also represented by the teachers) participated in the 
administrator focus group, including one technology coordinator. All four had attended RTI 
administrator meetings. 

Teachers 

Four of the five focus group teachers reported positive changes in their principals’ leadership, 
which they attributed to the RTI. The teachers reported increased discussion among their school 
staff, with more discussion occurring among elementary than secondary teachers. Teachers 
reported that the RTI generally fostered more connection among teachers within the schools than 
between the schools. The five teachers said that those who participated increased their use of the 
RTI instructional strategies, especially the elementary teachers. One teacher observed that 
although the RTI increased technology use among participating teachers, it did not overcome the 
resistance of those who were uncomfortable with it. 

Teachers liked the content of the RTI, and two reported that their schools plan to continue to use 
the RTI materials for professional development. These teachers liked the online discussion and 
felt it helped connect them to other teachers and to exchange ideas. They agreed that the 
discussion would be improved if teachers were grouped into elementary and secondary levels, 
with the latter further grouped by subject area. All five teachers said that for ease of scheduling, 
they would prefer to have one longer meeting each month instead of two. One teacher reported 
that secondary teachers had conflicts with the meetings due to after-school obligations, such as 
coaching. Teachers also mentioned that meetings should not be scheduled in May because they 
are too busy then to attend the classes. Some teachers observed that communication would be 
improved if e-mails regarding the classes were sent ahead of time and to more than one person at 
each school. They also suggested that class materials and PowerPoint presentations be available 
at least one week prior to each class. Some schools experienced technology problems at various 
times, and the teachers found this frustrating. They pointed out that it takes significant time to set 
up electronic connections and implement changes in small schools, where technology staff may 
be overburdened or non-existent. 

Administrators 

The four administrators agreed that the RTI provided them with new ideas about leadership. 
There was consensus that the RTI increased teacher discussion and improved instruction for the 
teachers who attended meetings and were attentive, but the administrators reported wide 
variation in their teachers’ level of engagement in the RTI. They expressed overall satisfaction 
with the content of the RTI, which in their view reinforced good instruction. They perceived the 
online delivery mechanism as valuable to rural schools. There was mixed experience with the 
technology connections, and they recommended a pilot test of the technology infrastructure prior 
to the first class meeting. 
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The administrators made suggestions for improving teacher engagement, which was an important 
issue for them. One suggestion was to increase teacher buy-in with an initial face-to-face or 
online presentation to teachers about the RTI — letting teachers know what they will learn and 
how it can help them. Similar to the teachers’ focus group comments, there was a consensus that 
scheduling the meetings before or after the normal school day made it difficult for teachers to 
attend meetings. The administrators recommended one longer meeting each month, ideally on 
teacher in-service days. They commented that compensating teachers in some way might 
improve their attendance. The technology coordinator observed that the teachers appeared more 
engaged at the meetings that the principal attended than at those where there was not an 
administrator present. It was suggested that the principal be required to attend all teacher 
meetings and to take attendance. They also suggested that each school have a follow-up staff 
meeting shortly after each RTI teacher meeting to discuss what the teachers had learned. 

Summary of Perceived Intervention Effectiveness and Suggested Improvements 

There is evidence that, overall, administrators and teachers, particularly the high participation 
teachers, were satisfied with the RTI and its components. Analyses of survey data indicated 
teachers’ agreement with positive statements about the RTI, and high-participation RTI teachers 
agreed significantly more with these statements compared to low-participation RTI teachers. 
Administrators gave high ratings to the quality and usefulness of different RTI components with 
a mean of four on most items on a five-point scale (where five indicates “high” quality or 
usefulness). The only exception was the technology platform, which they rated as “neutral.” The 
teacher and administrator post-intervention focus groups and the mid-intervention administrator 
formative interviews suggest that participants valued the content of the RTI class meetings, and 
that teachers who participated learned from the meetings and the online discussions. The 
opportunity for online professional development was viewed as valuable to rural schools. 

The data revealed some logistical issues that made RTI participation difficult for some. 
Suggestions were made regarding meeting preparation, scheduling, and technology testing that 
might improve the quality of the RTI delivery. Survey, focus group, and interview data suggest 
that technical difficulties occasionally interfered with participants’ ability to hear, see, or attend 
online class meetings, and that participants found this frustrating. 

Teachers perceived the RTI as having a slight positive influence on their instructional practices. 
Mean responses for statements about the RTFs positive impact on teaching, technology 
integration, reflective dialogue, school support, and student use of technology were between 
“neutral” and “agree.” The treatment teachers reported that the RTI increased or improved six 
aspects of their teaching and their students’ learning, and the teachers with a high level of 
participation reported significantly more improvement than the teachers with a low level of 
participation. Data from the teacher focus group were consistent with these results. Administrator 
interview and survey data also support the conclusion that the RTI influenced instruction but 
suggest that teacher participation in RTI activities was an important mediating factor. Although 
RTI teachers reported on the post-intervention teacher survey that they participated between 
“some extent” and a “considerable extent,” discussion board records of participation, survey 
data, focus group data, and administrator interviews suggest that there was extensive variability 
in teacher participation and engagement in the RTI program. Administrators were particularly 
concerned about those teachers who were not engaged in the RTI. Given that the level of teacher 
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participation (high vs. low) mediated teachers’ perceptions about the RTI and its effects, efforts 
to improve teacher engagement might increase the overall effectiveness of the RTI at 
participating schools. 

On the satisfaction survey, RTI administrators reported that the RTI had positive influences on 
their leadership behaviors, particularly in the areas of focusing on school priorities, clarifying 
priorities as a school leader, and forming an organizational vision. Data from the administrator 
interview and the teacher focus group confirmed that the RTI positively influenced leadership. 

Other Results 

In addition to the findings for each of the six research questions, there are other results that 
inform the interpretation of these findings. Analyses were conducted on the amount of different 
types of professional development that the treatment and control teachers experienced the year 
prior to and during the RTI intervention. These data were collected using the teacher pre-post 
intervention survey. 

Teachers in the treatment and control groups did not statistically differ on the pre-survey in the 
amount of professional development they experienced the year prior to the start of the 
intervention. This included professional development required by the district, professional 
development at the school, and non-required professional development such as conferences. 
Importantly, the two groups did not differ on the pre-survey in participation in online 
professional development (“less than 8 hours to none”). On the post-survey, the control teachers 
were asked about their professional development for the year during which the RTI was 
implemented. The teachers in the treatment group were asked about their professional 
development other than the RTI. The teachers in the control group reported participating in 
significantly more district-required professional development than the teachers in the treatment 
group while the RTI was being implemented (p < .05), although the absolute amount was still 
small for both groups. (The means were 2.70 and 2.95 for treatment and control teachers 
respectively, on a response set of 2 = “8 hours or less” and 3 = “9-16 hours”). The two groups of 
teachers again did not differ in amount of non-RTI online professional development. 

These data on teachers’ participation in professional development other than the RTI are 
important for interpreting the findings of this study. The data indicate that the treatment and 
control teachers experienced similar amounts of professional development prior to the RTI 
intervention. They also demonstrate that treatment teachers’ non-RTI professional development 
was not significantly more than the control group’s. However, the control group did participate 
in slightly more district-required professional development than the treatment group, which 
theoretically could attenuate RTI effects. 

Another set of data analyses addressed possible influences of teacher demographic variables on 
study findings. As previously described, treatment teachers reported significantly more years of 
teaching experience and more years teaching at their current school than control teachers. To 
determine whether experience was related to study outcomes, Pearson correlations were 
computed for each of the two experience variables with each variable measured on the teacher 
pre-intervention survey (see Table 5). None of the correlations was statistically significant, 
indicating that teacher experience was unrelated to study outcomes, so additional sub-group 
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analyses (similar to those performed for levels of teacher participation in the RTI) were not 
conducted. 

There was some suggestion in the focus group data that teachers’ engagement in the RTI varied 
depending on whether teachers worked at the elementary or secondary level. To address this 
possibility, a Chi Square was calculated to determine whether elementary teachers were more 
likely than secondary teachers to be in the high RTI participation group. The result from this 
analysis was not statistically significant, indicating that teacher grade level did not influence 
reported participation in the RTI. In addition, Pearson correlations between grade level and each 
variable measured on the teacher pre-intervention survey (see Table 5) were not statistically 
significant. 



SUMMARY AND CONCLUSIONS 

This section summarizes and interprets the primary findings of McREL’s RTI study, presents 
conclusions about the effectiveness of the RTI, and provides recommendations for improvements 
in the intervention. 

Summary of Findings 

First, the RTI was implemented with fidelity. Meetings occurred as planned with few 
technological difficulties. However, the difficulties that did occur were frustrating for the 
participants, especially for those in schools that did not have dedicated technology support. 
Second, teacher participation in the RTI activities varied greatly, and the degree of teacher 
participation mediated many of the results. Third, although teachers responded at moderately 
high rates to the individual pre-surveys and post-surveys, many teachers did not complete both 
surveys. As a result, the statistical analyses that depended on pre-post matches had small sample 
sizes. The necessity to conduct sub-group analyses based on teachers’ levels of participation 
further reduced sample sizes. Several findings reported in the results section approached 
statistical significance and may have reached significance with a larger sample size. 

Overall, the findings were mixed for the six research questions. Based on pre-post measures of 
professional community, the RTI did not change the aspect of professional community related to 
shared norms and values reflected in statements such as, “Teachers share beliefs and values 
about what the central mission of this school should be.” However, the RTI did influence the 
extent of teachers’ collaborative activities. The RTI increased the reported frequency of both 
online and non-online teacher collaboration. The online collaboration obviously occurred 
through the RTI, but the frequency of non-online collaboration such as peer feedback also 
increased more for the RTI teachers than the control teachers, and the increase was greater for 
teachers who had a higher level of participation in the RTI. The RTI administrators who were 
interviewed and the RTI teachers in the focus group confirmed the increase in collaborative 
discussion within the schools. The content analysis of the threaded discussions provided 
evidence of online professional community among teachers from the different schools, 
particularly in their reflective dialogue on instruction. 

The RTI did not significantly influence the variables related to teacher retention in this study, 
which included teacher efficacy, job satisfaction, and intention to stay in teaching. Interventions 
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that attempt to change teachers’ attitudes toward teaching generally produce relatively small 
effect sizes that require large sample sizes to detect (Lipsey & Wilson, 1993). Another problem 
was the possible disengagement of the low RTI teachers, who had the lowest mean response 
regarding intent to return to the current teaching assignment. 

The analysis of factors that influence teachers’ decisions about whether to continue teaching 
produced some interesting findings. Treatment and control teachers did not differ in their 
perspectives that the students, the rural lifestyle, and the location close to family are strong 
influences on the continuation of their teaching careers. However, the high RTI group perceived 
professional development and the influence of other teachers in the school as less important 
factors influencing their decisions to continue teaching after the RTI intervention, while the 
opposite occurred for the low RTI and control groups. This finding provides indirect evidence 
about RTI impact. 

Participation in the RTI did influence teachers’ reported use of the instructional strategies that 
comprise the content of the online teacher classes. The high group indicated significantly more 
frequent use of these strategies than the low RTI group. The administrators of RTI schools 
perceived increased use of RTI instructional strategies by over 50 percent of the teachers. There 
was a non-significant trend suggesting that RTI administrators perceived more improvement in 
teaching quality over the past school year compared to administrators in control schools. 

The findings related to technology were complicated by pre-existing differences in the sample of 
teachers who responded to the pre-post technology survey. The RTI teachers were significantly 
higher in perceptions of technology skill and school acceptance of technology constructs related 
to technology use in the school than the control teachers. This finding necessitated the use of the 
small matched sample of teachers completed both the pre-and the post-technology surveys. 
Findings based on this sample indicated that the RTI did not have statistically significant effects 
on increasing treatment teachers’ comfort with technology compared with the control sample: 
both groups experienced significant but similar increases. This finding may be related to the 
sample selection. The control schools were as interested in participating in the RTI as the 
treatment schools, which suggests that the control schools would continue in their use of 
technology and attempts to integrate technology with or without the RTI. 

Some indication of RTI influence on teachers’ use of technology was found in the post- 
technology survey completed by the administrators. The RTI administrators perceived moderate 
positive effects of the RTI on technology integration. In interviews, about half of the 
administrators perceived increased technology comfort and use in the classroom, but the others 
said their teachers were already comfortable with technology prior to the RTI. This suggests that 
there may have been a ceiling effect with regard to technology in some of the schools in the 
study. 

Regarding the influence of the RTI on leadership, the data indicated some differences between 
the perceptions of teachers and administrators. The RTI did not significantly change treatment 
teachers’ perceptions of their principals’ leadership relative to perceptions of control teachers. 
However, on the post-technology survey, the RTI administrators had higher mean perceptions 
related to their technology leadership compared to control administrators, particularly in their 
reported awareness of how to help teachers integrate technology. However, there are no pre- 
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survey data on technology leadership, so these group differences might have existed prior to the 
RTI intervention. 

The question concerning perceptions of intervention effectiveness and suggestions for 
improvement is primarily an evaluation question. While the data are only from RTI participants, 
the comparison of perceptions of teachers with high versus low levels of participation provides a 
way to assess intervention effectiveness. In addition, the combination of quantitative and 
qualitative data lends strength to the findings. 

The RTI teachers who participated more in the intervention reported greater impact from the RTI 
on teaching and student learning than the teachers who participated less, although the level of 
impact was moderate. Similarly those teachers who participated more expressed more 
satisfaction with the various components of the RTI. Administrators also indicated their 
satisfaction with the RTI and rated the various components of the intervention as high in utility 
and usefulness. RTI administrators perceived positive influences of the RTI on their leadership, 
particularly in the areas of focusing on school priorities, clarifying priorities as a school leader, 
and forming an organizational vision. Data from the administrator interviews and the teacher 
focus group also suggest a positive RTI influence on leadership, although teacher survey results 
did not support this observation. 

Administrators were concerned about those teachers who were not engaged in the RTI. They 
observed that those who participated learned from the meetings and the online discussions and 
that the RTI influenced instruction. The statistical analyses comparing teachers who had high 
versus low levels of participation confirm the administrators’ observations. Both administrators 
and teachers made suggestions on how to improve the RTI intervention. 

Conclusions 

Together the various sources of data and the analyses of these data lead to several conclusions. 
The major findings are: 

• The RTI had a positive influence on teacher collaboration and the 
development of professional community. 

• The RTI did not influence teachers’ perceptions of their efficacy for teaching, 
job satisfaction, or school environment. 

• Teachers who participated more in the RTI benefited more in terms of 
instructional improvements. 

• There was limited evidence that the RTI increased technology use, mainly 
because several of the schools that participated were already technologically 
proficient. 

• Teachers did not perceive changes in their principals’ leadership, but 
administrators perceived positive influences of the RTI on their leadership 
practices. 
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• Teachers and administrators were satisfied with the content and delivery of 
the RTI professional development and suggested recommendations for 
improvement. 

Recommendations 

Recommendations for improvement of the RTI in are in the areas of scheduling, participant 

information, discussion boards, and logistics. 

With regard to scheduling: 

• Change the two monthly RTI teacher class meetings to one longer monthly 
meeting. 

• Attempt to schedule the monthly teacher meeting during the school day. If this 
is not possible, consider offering teachers compensation of some type. 

• Do not schedule class meetings in May because teachers are too busy then to 
attend. 

With regard to participant information: 

• Provide each teacher with the book on which the content is based. 

• Provide teachers with incentives to participate, such as college credit. 

• Give teachers more information about the RTI at the beginning of the program 
to increase their buy-in. 

• Make all requirements for participation clear to teachers at the start of the 
program. 

• Provide administrators with guidance on how to motivate their teachers and 
monitor their participation. 

• Clarify administrators’ responsibilities. 

• Require an administrator to attend every teacher meeting. 

• Suggest that each RTI meeting be followed shortly afterward by a staff 
meeting. 

With regard to discussion boards: 

• Communicate the requirements regarding suggested levels of discussion board 
participation to administrators and teachers at the start of the program. 

• Divide the threaded discussions into elementary and secondary teachers; 
consider further dividing the secondary level by subject area. 
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With regard to logistics: 



• E-mail the contact person several days before the meetings. 

• E-mail more than one contact person in case the main contact person is 
unavailable. 

• Send the PowerPoint presentations to schools several days before the class 
meetings. 

• Increase testing time for the initial technology setup and for technology 
changes. 
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